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Continued Examination 

1 . In accordance with MPEP 1214.04 [R-3], If the examiner has specific knowledge 
of the existence of a particular reference or references which indicate nonpatentability of 
any of the appealed claims as to which the examiner was reversed, he or she should 
submit the matter to the Technology Center (TO Director for authorization to reopen 
prosecution under 37 CFR 1.198 for the purpose of entering the new rejection. See 
MPEP § 1002.02(c) and MPEP g 1214.07. The TC Director's approval is placed on the 
action reopening prosecution ". The examiner notes that the Board of Appeals has 
presented the existence of evidence regarding the nonpatentability of the claims. 

With respect to the decision of the Board of Appeals dated 8/14/2007, the Board 
of Appeals has indicated that while Reimer alone fails to teach the "ascertaining" and 
"prompting" limitations of the independent claims (see Page 9, Second Paragraph of 
the Board Decision), the Board has clearly indicated that either Menand or Wang 
teach the "ascertaining" and "prompting" limitations (see Page 10, First Paragraph for 
the Board noting that Wang teaches the limitation and Page 10, Third Paragraph 
for the Board noting that Wang discloses the limitations) . Prosecution on the 
merits of this application is reopened on all claims, which is considered unpatentable for 
the reasons indicated by the Board of Appeals. 

Claim Rejections - 35 (JSC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

2. Claims 1-25, 27-52 and 54-59 are rejected under 35 U.S.C. 103(a) as being 

unpatentable over Reimer et al. (U.S. Patent No. 5,553,221) in view of Wang et al. (U.S. 

Patent No. 6,766,320). 

Referring to claim 1, Reimer discloses providing video query processing software 
(see Column 8, Lines 54-59 for software 210 that enables the processor 204 
perform functions described herein , and Column 16, Lines 26-28 for the user 
being able to send a query while viewing/interacting with a movie). Therefore, the 
user is provided video query processing software at the user device 106. 

Reimer also discloses providing video content (see Column 7, Lines 58-63 for 
the user at the user device 106 receiving foundation information and Column 7, 
Lines 2-6 for the foundation information being "video content"). The examiner 
notes that the limitation "video content" is broad and could encompass audio, video or 
data (content) associated with a video program. 

Reimer also discloses dynamically linking the software to the video content (see 
Column 16, Lines 26-28 for the user at the user device 106 interacting with the 
video by sending a query at anytime while viewing and/or interacting with a 
movie). Note that the specification of the instant application defines "dynamically 
linked" as being able to interact with the video content and associated audio and text, in 
real time. Therefore, since the user can interact with the video by sending a query, the 
user device's (106) software (210) is dynamically linked to the video content 



Application/Control Number: 09/818,303 Page 4 

Art Unit: 2623 

Reimer also discloses receiving by the software a query keyed to a segment of 
the video content (see Column 16, Lines 28-41 for pausing the video content to a 
specific frame of the movie and the user indicating a question about the movie 
related to the movie, scene, cut and/or frame that is currently being displayed). 

Reimer also discloses determining by the software an answer to the query (see 
Column 18, Lines 1-2 for the user providing a query asking "Who is this?" and 
Column 18, Lines 65-67 and Column 19, Lines 1-3 for sending an answer to the 
question asked by the user). 

As noted by the Board of Appeals on Page 9, Reimer does not meet the 
requirements of the "ascertaining" and "prompting" limitations. 

The Board of Appeals further notes that Wang at Column 14, teaches confirming 
the intent of the user query when it is not fully apprehended by the natural language 
interpreter, thereby rendering claim 3 (dependent claim) obvious, which in turn, would 
render claim 1 obvious and therefore teaches the limitations void from the Reimer 
reference. 

After further review of the teachings of Wang at Column 14, the examiner 
specifically notes Column 14, Lines 24-26 and 42-45 for Wang clearly teaching that the 
search system provides "ascertaining if the query needs to be recast and prompting for 
user input if the query needs to be recasf. The examiner further notes Column 1 5, 
Lines 10-24 for a specific example of the teachings of Column 14 noted above. 

At the time the invention was made, it would have been obvious to a person of 
ordinary skill in the art, to modify the query system, as taught by Reimer, by including 
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the ascertaining and prompting features, as taught by Wang, for the purpose of 
improving the efficiency and accuracy in the information retrieval system based on a 
user's search intention (see Column 14, Lines 16-18 of Wang). 

Referring to claim 2, Reimer discloses that the software is within a video 
processing system (see Column 7, Lines 62-63 for the user device 106 presenting 
foundation information, where the foundation information is a movie (see Column 
7, Lines 2-3 for the foundation information being video content)). Therefore, since 
the user device 106 can receive and present a movie, the user device 106 is a video 
processing system, that contains the software within the video processing system (see 
the rejection of claim 1). 

Referring to claim 3, Wang discloses a user query system (see Column 4, Lines 
45-47) in a video processing system (see Column 4, Lines 54-55 for the client device 
102 being a set-top box), and that the user query system specifically uses the Internet 
(see Column 4, Line 47). 

Referring to claim 4, the applicant's specification of the instant application defines 
a service mode to be when the video processing system is operating with an external 
database, which has access to a database other than the Internet (e.g. access to a 
database of a remote server) (see Page 8, Lines 18-19 of the Applicant's 
specification). 
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Reimer discloses that the foundation information database is a remote database 
(see Column 7, Lines 8-11 for the foundation information database 112 being a file 
server and Figure 1 for the user device 106 communicating with the foundation 
information database 112 through a communication medium 124, and is therefore 
a remote server). Therefore, the video processing system is operating in a service 
mode. 

Referring to claim 5, Reimer discloses providing video content includes providing 
video content in real-time (see Column 16, Lines 26-28 for allowing a user to send a 
query to the presentation and control component 104 at any time while viewing 

the movie). Therefore, since the user may send a query while viewing the movie, the 
video (that the viewer is watching) is provided in real-time. Also note Column 6, Line 67 
and Column 7, Line 1 for providing foundation information (video content) on-demand, 
therefore, since the answer to the query is provided on-demand in the form of 
foundation information, the on-demand data transmitted to the user is in "real-time". 

Referring to claim 6, Reimer discloses that the foundation information, stored in 
the foundation information database 112 (see Column 7, Lines 10-11) is movie data 
(video content) and can be provided to a user on-demand (see Column 6, Line 67 and 
Column 7, Line 1). Also note that source information (Figure 3) used for creating the 
foundation information includes pre-production, production and post production 
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information, which is used to create the foundation information, stored in the foundation 
information database 112 (see Column 9, Line 40 through Column 10, Line 15). 

Referring to claim 7, Reimer discloses receiving information by the software (see 
again Column 18, Lines 65-67 and Column 19, Lines 1-3 for receiving information 
at the user device 106 (which contains the software that operates the user device) 
in the rejection of claim 1), wherein the information is derived from a database (see 
Column 18, Lines 45-58 for deriving actors names from an Actor In Take Table 
1002 that relate to the user's query), and wherein the information answers the query 
(see Column 18, Lines 65-67 and Column 19, Lines 1-3 for the information being 
the actors stored in the table, which is transmitted to the viewer in order to 
answer the viewer's query). 

Referring to claim 8, Reimer discloses receiving data from the database, wherein 
the data includes the information (see Column 20, Lines 22-26 for receiving data 
equal to the "Director" section of the table, where the data includes the 
information sought by the user query). 

Reimer also discloses extracting the information from the data (see Column 20, 
Lines 27-29 for extracting the name of the director from the person column of the 
row). 
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Referring to claim 9, Reimer discloses finding data in the database, wherein the 
data includes the information and extracting the information from the data at the 
database (see Column 18, Lines 45-58 for retrieving information from data in the 
Actor In Table 1002 database). The examiner notes that in order to retrieve the data 
(which includes the information) in the database, it must inherently find the proper data 
in order to extract (retrieve) the data corresponding to the user's query. 

Reimer also discloses sending the information to the software (see Column 18, 
Lines 65-67 and Column 19, Lines 1-3 for sending the extracted information to the 
user device 106, which contains the software used to control the user device 106 
(see the rejection of claim 1)). 

Referring to claim 10, Reimer discloses identifying the database (different 
tables located in the databases (see Figures 5-6)) by a pointer (see Column 7, 
Lines 25-30 and Figures 5-6 for accessing indices of the foundation information) 
located in a search site descriptions repository (see Column 7, Lines 21-30 for an 
index interface component 118, which controls access to index information 310 in 
an index information database 122 that references and organizes the data stored 
in the foundation information database 112). The examiner notes that the search 
site descriptions repository limitation is met by the index information database 122, but 
is not limited thereto. 
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Referring to claim 1 1 , Reimer discloses that the software is within a video 
processing system (see the rejection of claim 2), and wherein the database is external 
to the video processing system (see Figure 1 for the user device 106 being external 
to the index information database 122 and communicate through a 
communications medium 124, which can be a wide area network (see Column 6, 
Lines 47-48)). 

Referring to claim 12, see the rejection of claim 3. 

Referring to claim 13, Reimer discloses that the database is coupled to a remote 
server (see Index information database 122 connected to presentation and control 
component(s) 104 remotely (through communications network 124) in Figure 1). 

Referring to claim 14, Reimer discloses that providing video content includes 
providing the dynamic video content to a user of the video query processing method 
(see again Column 7, Lines 58-63 for providing the dynamic video content, 
wherein the video content is dynamic because the results of the query are going 
to change depending on the type of question asked), and wherein receiving the 
query includes communicating the query to the software by the user (see Column 7, 
Lines 52-53 for the user devices 106 receiving input from the user regarding what 
question to ask the system). 



Application/Control Number: 09/818,303 Page 10 

Art Unit: 2623 

Referring to claim 15, Reimer discloses communicating the answer to the user 
(see Column 18, Lines 65-67 and Column 19, Lines 1-3). 

Referring to claim 16, Reimer discloses receiving by the software, information 
derived from each database of a plurality of databases (see Figure 11 for accessing 
multiple tables (databases) and presenting the queried information to the user), 
wherein each database is external to the video processing system (see Figure 1, 
where the user device is external (remote) to the Index Information Database(s) 
122 and Foundation Information Database(s) 112 and Column 6, Lines 43-56) and 
wherein the information derived from each database partially answers the query (see 
Column 19, Lines 55-67 and Column 20, Lines 1-37 for the system answering the , 
user's query, "What Other Films Has This Director Worked On?", which accesses 
multiple databases (1202-1216), where each database partially answers the query 
(see Column 20, Lines 5-9, 18-21 and 22-33 for each database pulling certain 
information (which therefore inherently partially answers the question) to display 
to the user)). 

Reimer also discloses merging the information derived from each database to 
arrive at the answer (see Column 20, Lines 33-37 for displaying the results derived 
from the information pulled from databases 1202-1216 previously discussed). 

Referring to claim 17, see the rejection of claim 16 (also note the rejection of 
claim 8). Note that the data received from each database inherently includes the 
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information. For example, note Column 20, Lines 2-3 for person data (the data) being 
received from the One Movie Credit database 1202, which contains person information . 

Referring to claim 18, see the rejection of claims 16-17 (also note the rejection of 
claim 9). Further note that the extracting is executed at each database (see again 
Column 20, Lines 22-34 for exacting the data from the database). Further note that 
Reimer teaches sending the information derived from each database (the answer to the 
query) to the software (see Column 20, Lines 34-37 for sending the information to 
the user device, which contains the query software that allows a user to make a 
query). 

Referring to claim 19, Reimer discloses that the query received by the software is 
a canned (pre-stored, as defined by Applicant's specification) query (see Column 16, 
Lines 43-46 for the user selecting the query from a menu, thereby providing a list 
of questions that are already stored in a list for presentation to the user). 

Referring to claim 20, Reimer discloses that the canned query is a function of a 
genre of the video (see Column 16, Lines 38-39 for the user's query being, "What 
other movies have script lines similar to what was just said?", therefore providing 
a query that requests other movies that relate to the type of movie (genre) the 
viewer is watching). 
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Referring to claim 21, Reimer discloses that the query received by the software is 
an unbounded query (see Column 16, Lines 62-65 for asking the question "Who is 
this person?", which could pertain to multiple actors on a screen) and further 
deriving the at least one canned (pre-stored) query from the unbounded query (see 
Column 16, Lines 55-67 and Column 17, Lines 1-9 for deriving the question, "Who 
is the character in this scene?"). 

Referring to claim 22, Reimer discloses that the query received by the software is 
in indefinite form, and wherein ascertaining further comprises recasting the received 
query in definite form (see the rejection of claim 21 for deriving a canned query 
from an unbounded query, where the indefinite for of the question is "Who is this 
person?" and recasting the question to a definite, more precise question, "Who is 
the character in this scene?"). Also note the rejection of claim 1 and the arguments 
above for further covering the ascertaining and recasting limitations. 

Referring to claim 23, Reimer discloses receiving by the software a program-level 
question in relation to the video content (see Column 16, Line 36 for the question, 
"What other movies has the director done?). The examiner notes that since this 
question relates to other movies/programs, that the question is therefore, a program- 
level question. 

Reimer also discloses ascertaining by the software an answer to the question 
(see Column 20, Lines 35-37). 
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Referring to claim 24, Reimer discloses extracting features from the video 
content (see Column 10, Lines 65-67 and Column 11, Lines 1-19 for capturing the 
source information (video content) and extracting features from the source 
information into the foundation information 306 (Figure 3, which is stored in 
foundation database 112) and processes index information from the extracted 
features at Column 11, Lines 32-34 and Lines 53-59), wherein the ascertaining 
includes utilizing the extracted features to answer the question (see Column 14, Lines 
51-55 for using the extracted data to answer user queries). 

Referring to claim 25, Reimer discloses storing the extracted features in transient 
memory prior to utilizing the extracted features (see Column 7, Lines 10-11 and 
Column 8, Lines 43-63) to answer the question (see the rejection of claim 24). 

Referring to claim 27, Reimer disclose that extracting features from the video 
content includes extracting features from the video program of the video content (see 
Column 10, Lines 65-67 for extracting the source information 302 and Figure 4 
and Column 10, Lines 11-15 for the source information 302 containing a movie 
(video program)). 
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Referring to claims 28-52 and 54, see the rejection of claims 1-25 and 27, 
respectively. Note Figures 1 and 2 for the system to execute the set forth by the 

method/process rejected in claim 1 . 

Referring to claim 55, see the rejection of claim 1 . Also note that Reimer teaches 
a processor (Figure 2, 204), memory structure coupled to the processor (primary 
memory 208 coupled to the processor 204 though bus 206 in Figure 2), a local 
database coupled to the processor (secondary memory 214 coupled to the 
processor 204 through bus 206 in Figure 2), a video input device coupled to the 
processor and the local database (see Column 6, lines 36-37 for a STB which is 
inherently coupled to the processor in order to process the query specified by the 
user), a user input device coupled to the processor (see Column 6, lines 38-39 for a 
keyboard, which is inherently coupled to the processor in order to enter the 
user's query) and an output device coupled to the processor (see Column 6, Lines 
36-37 for a television monitor, which would inherently be coupled to the 
processor 204 and local database 214 in order to display the query answers to a 
user disclosed by Reimer). 

Referring to claim 56, see the rejection of claim 16. 

Referring to claim 57, Reimer discloses a video source (see Source Information 
302 in Figures 3-4), wherein the video processing architecture is configured to enable 
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the video source to transmit the video content to the video processing system (see 
Figure 1 for the Foundation Information Database(s) 112 being coupled to the 
user devices 106 (and devices 104, 108, 114, 116, 118 and 122) through 
communication medium 124, therefore the system would inherently transmit any 
source information to the proper components to process the video content in 
various ways). 

Referring to claims 58, Reimer discloses that the software is configured to 
receive the query from a user of the software (see the rejection of claim 1 and 
Figures 9A and 11 and the computer system containing the software in Figure 2). 

Referring to claim 59, note that the system disclosed in the rejection of claim 1 
includes a computer program product comprising a computer readable medium (see 
Figure 2) having a computer readable code embedded therein (see the control logic 
210 and data 212 in Figure 2). 

3. Claims 26 and 53 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Reimer et al. (U.S. Patent No. 5,553,221) in view of Wang et al. (U.S. Patent No. 
6,766,320) in view of Menard et al. (U.S. Patent No. 6,061 ,056). 

Referring to claim 26, Reimer and Wang discloses all of the limitations in claim 
24, as well as extracting features from the video content (see the rejection of claim 
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24), but fails to teach taking into account preferences of a user of the query processing 
method. 

Menard discloses a system which captures audio, video and closed captioning 
text data (similar to Reimer's capture and digitizer component 304), which allows a 
user to make an SQL type query to extract video, which takes into account the 
preferences of a user (see Column 4, Lines 63-65 and Column 6, Lines 28-56). 

At the time the invention was made, it would have been obvious to a person of 
ordinary skill in the art, to modify the capture system 304, as taught by Reimer and 
Wang, using the viewer preference capture system, as taught by Menard, for the 
purpose of allowing filtering and querying capability of live broadcasts and multimedia 
databases (see Column 3, Lines 16-20 of Menard). 

Referring to claim 53, see the rejection of claim 26. 

Conclusion 

4. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jason P. Salce whose telephone number is (571) 272- 
7301. The examiner can normally be reached on M-F 9am-6pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Miller can be reached on (571) 272-7353. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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